Augmented Video Processing

ABSTRACT

Embodiments described herein may allow for dynamic image processing based on biometric data. An example device may include: an interface configured to receive video data that is generated by an image capture device; an interface configured to receive biometric data of a user of the image capture device from one or more sensors generated synchronously with the video data; and an image processing system configured to apply image processing to the video data to generate edited video data. The image processing may be based, at least in part, on the biometric data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/534,980, filed Nov. 6, 2014, which claims priority to U.S.Provisional Application No. 61/933,060, filed Jan. 29, 2014, both ofwhich are hereby incorporated by reference in their entirety and for allpurposes. The application is also a continuation-in-part of U.S.application Ser. No. 14/535,037, filed on Nov. 6, 2014, which is herebyincorporated by reference in its entirety and for all purposes.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Computing devices such as personal computers, laptop computers, tabletcomputers, cellular phones, and countless types of Internet-capabledevices are increasingly prevalent in numerous aspects of modern life.Over time, the manner in which these devices are providing informationto users is becoming more intelligent, more efficient, more intuitive,or less obtrusive.

The trend toward miniaturization of computing hardware, peripherals, aswell as of sensors, detectors, and image and audio processors, amongother technologies, has helped open up a field sometimes referred to as“wearable computing.” In the area of image and visual processing andproduction, in particular, it has become possible to consider wearabledisplays that place a graphic display close enough to a wearer's (oruser's) eye(s) such that the displayed image appears substantiallyfurther away, such as might be displayed on a traditional image displaydevice. The relevant technology may be referred to as “near-eyedisplays.”

Wearable computing devices with near-eye displays may also be referredto as “head-mountable displays” (HMDs), “head-mounted displays,”“head-mounted devices,” or “head-mountable devices.” A head-mountabledisplay places a graphic display or displays close to one or both eyesof a wearer. To generate the images on a display, a computer processingsystem may be used. Such displays may occupy a wearer's entire field ofview, or occupy only a portion of a wearer's field of view. Further,head-mounted displays may vary in size, taking a smaller form such as aglasses-style display or a larger form such as a helmet, for example.

Emerging and anticipated uses of wearable displays include applicationsin which users interact in real time with an augmented or virtualreality. Such applications can be mission-critical or safety-critical,such as in a public safety or aviation setting. The applications canalso be recreational, such as interactive gaming. Many otherapplications are also possible.

HMDs may allow users to easily capture visual images and soundscollected from their environment, in some cases, completely hands free.The head- or body-mounted nature of the image capture device allows theviewer of the resulting images to visually experience the scene from thesame perspective as the user of the HMD. HMD-captured images and soundsmay also be shared with others in a variety of ways. Image and audiodata may be collected by the HMD and shared with others in real time orrecorded for immediate or later playback. The HMD may also transmitcaptured data to a remote computing device, such as a laptop, tablet,smartphone, or the cloud for further processing.

SUMMARY

An example device may include an image processing system configured toapply image processing to captured video data based, at least in part,on biometric data collected from a user of the image capture device. Theimage processing may be designed to enhance the viewing experience ofthe captured video. For example, the device may be configured to applyslow motion or blur effects to the captured video data based on thebiometric data. The device may include, or may be configured to receivebiometric data from, one or more sensors. In some examples, the imagecapture device may be included in a wearable device. The biometric datamay include, without limitation, heart rate, respiration rate and levelof movement of the wearer of the device.

In one aspect, embodiments of the present disclosure provide a computingdevice including: (1) an interface configured to receive video data thatis generated by an image capture device; (2) an interface configured toreceive, from one or more sensors, biometric data of a user of thedevice, wherein the biometric data is generated synchronously with thevideo data; and (3) an image processing system configured to apply imageprocessing to the video data to generate edited video data, wherein theimage processing is based, at least in part, on the biometric data.

Further embodiments of the present disclosure provide a methodincluding: (1) receiving, by a computing device, video data generated byan image capture device; (2) receiving, by a computing device, biometricdata of a user of the image capture device from one or more sensors,wherein the biometric data is generated synchronously with the videodata; and (3) applying image processing to the video data to generateedited video data, wherein the image processing is based, at least inpart, on the biometric data.

Some embodiments of the present disclosure provide a non-transitorycomputer readable medium having stored therein instructions executableby a processor to cause a computing device to perform functionsincluding: (1) receiving video data generated by an image capturedevice; (2) receiving, from one or more sensors, biometric data of auser of the image capture device, wherein the biometric data isgenerated synchronously with the video data; and (3) applying imageprocessing to the video data to generate edited video data, wherein theimage processing is based, at least in part, on the biometric data.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description, with reference where appropriate to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a wearable computing system according to an exampleembodiment.

FIG. 1B illustrates an alternate view of the wearable computing deviceillustrated in FIG. 1A.

FIG. 1C illustrates another wearable computing system according to anexample embodiment.

FIG. 1D illustrates another wearable computing system according to anexample embodiment.

FIG. 1E is a simplified illustration of the wearable computing systemshown in FIG. 1D, being worn by a wearer.

FIG. 1F is a simplified illustration of the wearable computing systemshown in FIG. 1D, being worn by a wearer.

FIG. 1G is a simplified illustration of the wearable computing systemshown in FIG. 1D, being worn by a wearer.

FIG. 2 is a simplified block diagram of a computing device according toan example embodiment.

FIG. 3 is a simplified block diagram of a system configured to captureand process video data according to an example embodiment.

FIG. 4 is a screenshot of an example user interface showing edited videodata in accordance to an example embodiment.

FIG. 5 is a flow chart of an example method, in accordance with anexample embodiment.

FIG. 6 is a flow chart of an example method, in accordance with anexample embodiment.

DETAILED DESCRIPTION

Example methods and systems are described herein. It should beunderstood that the words “example,” “exemplary,” and “illustrative” areused herein to mean “serving as an example, instance, or illustration.”Any embodiment or feature described herein as being an “example,” being“exemplary,” or being “illustrative” is not necessarily to be construedas preferred or advantageous over other embodiments or features. Theexample embodiments described herein are not meant to be limiting. Itwill be readily understood that the aspects of the present disclosure,as generally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations, all of which are explicitlycontemplated herein.

I. OVERVIEW

While HMDs may capture images and sounds from the viewing perspective ofthe wearer or operator, the raw image or audio data may not sufficientlyconvey the full range of emotions, stimulations, or sensations,experienced by the user, to a viewer of the recorded images. To improvethe raw video captured by an HMD, video editing and special effects(slow or fast motion scenes, pauses, cuts, timed music, coloring, etc.)can be used to better evoke the feelings and sensations of the moment ofcapture.

Biometric data sensed from the wearer of an HMD or biometric dataotherwise received by the HMD may be used to augment both front-endimage capture and back-end image processing, potentially in real time.In an example system, an HMD synchronously records image, audio, andbiometric data for automatic or dynamic recording or editing.

In one aspect, the received data may be used to control image capturesoftware or hardware on an HMD. For example, data taken from biometricsensors, such as heart rate, may be used to control the HMD's camera orvideo software or hardware, such as by controlling the speed orresolution of image capture. In some embodiments, data received from aninertial measurement unit (“IMU”) may cause the video camera to increaseits rate of frame capture or resolution when it detects that it isfalling. This portion of increased frame rate video may later be used tocreate a “slow motion” section of video during free-fall.

In another aspect, biometric data received from biometric, contextual or“movement” sensors may be used for assisted post-capture image or audioediting. Metadata from the sensors and other inputs may be synchronouslyrecorded with the image and audio data and may be used post-capture toautomatically edit or suggest edits to the image and audio data tocreate a more interesting or sensory-representative composition. Forexample, data collected from a heart-rate monitor (HRM) indicating anincrease in the wearer's heart rate, may automatically (or suggest thatthe user) slow-down that segment of recorded video to create visualemphasis.

Biometric data may include any data related to the sensory,physiological, behavioral, health, movement, context or other measurablecharacteristics of a living subject or a function of the living subject,such as heart rate, pulse rate, respiration rate, body temperature,perspiration, eye movements, blinking, muscle flinches or tension,strength of hand grip, etc. Biometric data may be gathered from directmeasurement of an aspect or characteristic of the body, or any dataindicative of the movement or context of a body. The system may alsoreceive certain contextual data, including GPS (or other location) data,elevation, type of movement (i.e. walking, driving, biking, swimming,etc.), weather, ambient temperature, etc. Further, motion-related datamay include speed, cadence or direction of movement of the body,orientation of the body, gravitational and inertial forces acting on thebody, etc.

Data may be collected from biometric, movement and contextual sensorsintegrated on the HMD, by sensors remote to the HMD (such as biometricsensors placed on other portions of the body or in communication withthe body, or from a remote GPS-ready device such as a smartphone), byother computing devices remote from the HMD (such as a remote devicesuch as a smartphone having location tracking and internetcapabilities), or by manual input by the wearer of the HMD. Biometricsensors may include heart rate monitors, pulse oximeters, thermometers,galvanic skin response sensors, microphones, bone conductingtransducers, Doppler devices, eye tracking devices, electromyographs(“EMG”), strain gauges, etc. Movement-sensors may includeaccelerometers, IMUs, infrared sensors, ultrasonic sensors, odometers,pedometers, etc. and may also detect position and orientation withoutnecessarily detecting “movement.” Contextual sensors may include GPSdevices, thermometers, and computing devices which may receive inputsfrom the wearer of the HMD.

The biometric data may be used to augment real-time image capture andediting of images captured with, for example, an HMD. Certain biometricor motion data, such as a spike in heart rate or a sudden increase ordecrease in an accelerometer reading, may be used as a cue to: (a)adjust image-capture hardware settings on HMD to, for example, increaseframe rate or resolution (in anticipation of some post-capture editing),(b) begin a rolling video buffer in case the image-capture hardware wasnot recording, or (c) suggest certain edits or automatically edit thecorresponding video data in post-production. Post-capture editing canoccur off-device (e.g., in the cloud) using the sensor or input datauploaded along with the image and audio data.

A variety of editing techniques, including scene cuts, fade-ins,fade-outs, playback speed, music, color, light intensity, etc., may besynchronized with or used to enhance the captured image or audio data tocorrelate to suspense, stress, anticipation, movement, intensity, etc.Thus, the playback speed of certain video segments may be edited tocorrespond to the wearer's in-the-moment experiences. Data reflecting alack of movement of the user and relatively low heart rate during acertain period may be interpreted as a relatively un-interesting videosegment and may be automatically cut or sped-up or a recommendation tocut or speed-up this segment may be made by the system. On the otherhand, data reflecting a high level of movement of the user, an elevatedheart rate, an elevated respiration rate, increased perspiration, or afixed eye gaze, etc. may be interpreted as a particularly exciting,interesting, or intense and, therefore, high intensity or high volumemusic, slow motion playback, color enhancement, etc. may be introducedor suggested. Further, the timing of the edits, such as the music's timesignature and tempo or the frequency of light or color intensityflashing in the visual data, may be chosen to match and may besynchronized with the cadence of the wearer's steps, peddling, ormovement or to the intensity of the wearer's physical movements.

It should be understood that the above embodiments and others describedherein are provided for purposes of illustration, and are not intendedto be limiting. Variations on the above embodiments and otherembodiments are possible, without departing from the scope of theinvention as set forth by the claims.

II. EXAMPLE WEARABLE COMPUTING DEVICES

Systems and devices in which example embodiments may be implemented willnow be described in greater detail. In general, an example system may beimplemented in or may take the form of a wearable computer (alsoreferred to as a wearable computing device). In an example embodiment, awearable computer takes the form of or includes a head-mountable device(HMD).

An example system may also be implemented in or take the form of otherdevices, such as a mobile phone, among other possibilities. Further, anexample system may take the form of non-transitory computer readablemedium, which has program instructions stored thereon that areexecutable by a processor to provide the functionality described herein.An example system may also take the form of a device such as a wearablecomputer or mobile phone, or a subsystem of such a device, whichincludes such a non-transitory computer readable medium having suchprogram instructions stored thereon.

An HMD may generally be any display device that is capable of being wornon the head and places a display in front of one or both eyes of thewearer. An HMD may take various forms such as a helmet or eyeglasses. Assuch, references to “eyeglasses” or a “glasses-style” HMD should beunderstood to refer to an HMD that has a glasses-like frame so that itcan be worn on the head. Further, example embodiments may be implementedby or in association with an HMD with a single display or with twodisplays, which may be referred to as a “monocular” HMD or a “binocular”HMD, respectively.

FIG. 1A illustrates a wearable computing system according to an exampleembodiment. In FIG. 1A, the wearable computing system takes the form ofa head-mountable device (HMD) 102 (which may also be referred to as ahead-mounted display). It should be understood, however, that examplesystems and devices may take the form of or be implemented within or inassociation with other types of devices, without departing from thescope of the invention. As illustrated in FIG. 1A, the HMD 102 includesframe elements including lens-frames 104, 106 and a center frame support108, lens elements 110, 112, and extending side-arms 114, 116. Thecenter frame support 108 and the extending side-arms 114, 116 areconfigured to secure the HMD 102 to a user's face via a user's nose andears, respectively.

Each of the frame elements 104, 106, and 108 and the extending side-arms114, 116 may be formed of a solid structure of plastic or metal, or maybe formed of a hollow structure of similar material so as to allowwiring and component interconnects to be internally routed through theHMD 102. Other materials may be possible as well.

One or more of each of the lens elements 110, 112 may be formed of anymaterial that can suitably display a projected image or graphic. Each ofthe lens elements 110, 112 may also be sufficiently transparent to allowa user to see through the lens element. Combining these two features ofthe lens elements may facilitate an augmented reality or heads-updisplay where the projected image or graphic is superimposed over areal-world view as perceived by the user through the lens elements.

The extending side-arms 114, 116 may each be projections that extendaway from the lens-frames 104, 106, respectively, and may be positionedbehind a user's ears to secure the HMD 102 to the user. The extendingside-arms 114, 116 may further secure the HMD 102 to the user byextending around a rear portion of the user's head. Additionally oralternatively, for example, the HMD 102 may connect to or be affixedwithin a head-mounted helmet structure. Other configurations for an HMDare also possible.

The HMD 102 may also include an on-board computing system 118, an imagecapture device 120, a sensor 122, and a finger-operable touch pad 124.The on-board computing system 118 is shown to be positioned on theextending side-arm 114 of the HMD 102; however, the on-board computingsystem 118 may be provided on other parts of the HMD 102 or may bepositioned remote from the HMD 102 (e.g., the on-board computing system118 could be wire- or wirelessly-connected to the HMD 102). The on-boardcomputing system 118 may include a processor and memory, for example.The on-board computing system 118 may be configured to receive andanalyze data from the image capture device 120 and the finger-operabletouch pad 124 (and possibly from other sensory devices, user interfaces,or both) and generate images for output by the lens elements 110 and112.

The image capture device 120 may be, for example, a camera that isconfigured to capture still images or to capture video. In theillustrated configuration, image capture device 120 is positioned on theextending side-arm 114 of the HMD 102; however, the image capture device120 may be provided on other parts of the HMD 102. The image capturedevice 120 may be configured to capture images at various resolutions orat different frame rates. Many image capture devices with a smallform-factor, such as the cameras used in mobile phones or webcams, forexample, may be incorporated into an example of the HMD 102.

Further, although FIG. 1A illustrates one image capture device 120, moreimage capture device may be used, and each may be configured to capturethe same view, or to capture different views. For example, the imagecapture device 120 may be forward facing to capture at least a portionof the real-world view perceived by the user. This forward facing imagecaptured by the image capture device 120 may then be used to generate anaugmented reality where computer generated images appear to interactwith or overlay the real-world view perceived by the user.

The sensor 122 is shown on the extending side-arm 116 of the HMD 102;however, the sensor 122 may be positioned on other parts of the HMD 102.For illustrative purposes, only one sensor 122 is shown. However, in anexample embodiment, the HMD 102 may include multiple sensors. Forexample, an HMD 102 may include sensors 102 such as one or moregyroscopes, one or more accelerometers, one or more magnetometers, oneor more light sensors, one or more infrared sensors, or one or moremicrophones. Other sensing devices may be included in addition or in thealternative to the sensors that are specifically identified herein.

The finger-operable touch pad 124 is shown on the extending side-arm 114of the HMD 102. However, the finger-operable touch pad 124 may bepositioned on other parts of the HMD 102. Also, more than onefinger-operable touch pad may be present on the HMD 102. Thefinger-operable touch pad 124 may be used by a user to input commands.The finger-operable touch pad 124 may sense at least one of a pressure,position or a movement of one or more fingers via capacitive sensing,resistance sensing, or a surface acoustic wave process, among otherpossibilities. The finger-operable touch pad 124 may be capable ofsensing movement of one or more fingers simultaneously, in addition tosensing movement in a direction parallel or planar to the pad surface,in a direction normal to the pad surface, or both, and may also becapable of sensing a level of pressure applied to the touch pad surface.In some embodiments, the finger-operable touch pad 124 may be formed ofone or more translucent or transparent insulating layers and one or moretranslucent or transparent conducting layers. Edges of thefinger-operable touch pad 124 may be formed to have a raised, indented,or roughened surface, so as to provide tactile feedback to a user whenthe user's finger reaches the edge, or other area, of thefinger-operable touch pad 124. If more than one finger-operable touchpad is present, each finger-operable touch pad may be operatedindependently, and may provide a different function.

In a further aspect, HMD 102 may be configured to receive user input invarious ways, in addition or in the alternative to user input receivedvia finger-operable touch pad 124. For example, on-board computingsystem 118 may implement a speech-to-text process and utilize a syntaxthat maps certain spoken commands to certain actions. In addition, HMD102 may include one or more microphones via which a wearer's speech maybe captured. Configured as such, HMD 102 may be operable to detectspoken commands and carry out various computing functions thatcorrespond to the spoken commands.

As another example, HMD 102 may interpret certain head-movements as userinput. For example, when HMD 102 is worn, HMD 102 may use one or moregyroscopes or one or more accelerometers to detect head movement. TheHMD 102 may then interpret certain head-movements as being user input,such as nodding, or looking up, down, left, or right. An HMD 102 couldalso pan or scroll through graphics in a display according to movement.Other types of actions may also be mapped to head movement.

As yet another example, HMD 102 may interpret certain gestures (e.g., bya wearer's hand or hands) as user input. For example, HMD 102 maycapture hand movements by analyzing image data from image capture device120, and initiate actions that are defined as corresponding to certainhand movements.

As a further example, HMD 102 may interpret eye movement as user input.In particular, HMD 102 may include one or more inward-facing imagecapture devices or one or more other inward-facing sensors (not shown)sense a user's eye movements or positioning. As such, certain eyemovements may be mapped to certain actions. For example, certain actionsmay be defined as corresponding to movement of the eye in a certaindirection, a blink, or a wink, among other possibilities.

HMD 102 also includes a speaker 125 for generating audio output. In oneexample, the speaker could be in the form of a bone conduction speaker,also referred to as a bone conduction transducer (BCT). Speaker 125 maybe, for example, a vibration transducer or an electroacoustic transducerthat produces sound in response to an electrical audio signal input. Theframe of HMD 102 may be designed such that when a user wears HMD 102,the speaker 125 contacts the wearer. Alternatively, speaker 125 may beembedded within the frame of HMD 102 and positioned such that, when theHMD 102 is worn, speaker 125 vibrates a portion of the frame thatcontacts the wearer. In either case, HMD 102 may be configured to sendan audio signal to speaker 125, so that vibration of the speaker may bedirectly or indirectly transferred to the bone structure of the wearer.When the vibrations travel through the bone structure to the bones inthe middle ear of the wearer, the wearer can interpret the vibrationsprovided by BCT 125 as sounds.

Various types of bone-conduction transducers (BCTs) may be implemented,depending upon the particular implementation. Generally, any componentthat is arranged to vibrate the HMD 102 may be incorporated as avibration transducer. Yet further it should be understood that an HMD102 may include a single speaker 125 or multiple speakers. In addition,the location(s) of speaker(s) on the HMD may vary, depending upon theimplementation. For example, a speaker may be located proximate to awearer's temple (as shown), behind the wearer's ear, proximate to thewearer's nose, or at any other location where the speaker 125 canvibrate the wearer's bone structure.

FIG. 1B illustrates an alternate view of the wearable computing deviceillustrated in FIG. 1A. As shown in FIG. 1B, the lens elements 110, 112may act as display elements. The HMD 102 may include a first projector128 coupled to an inside surface of the extending side-arm 116 andconfigured to project a display 130 onto an inside surface of the lenselement 112. Additionally or alternatively, a second projector 132 maybe coupled to an inside surface of the extending side-arm 114 andconfigured to project a display 134 onto an inside surface of the lenselement 110.

The lens elements 110, 112 may act as a combiner in a light projectionsystem and may include a coating that reflects the light projected ontothem from the projectors 128, 132. In some embodiments, a reflectivecoating may not be used (e.g., when the projectors 128, 132 are scanninglaser devices).

In alternative embodiments, other types of display elements may also beused. For example, the lens elements 110, 112 themselves may include: atransparent or semi-transparent matrix display, such as anelectroluminescent display or a liquid crystal display, one or morewaveguides for delivering an image to the user's eyes, or other opticalelements capable of delivering an in focus near-to-eye image to theuser. A corresponding display driver may be disposed within the frameelements 104, 106 for driving such a matrix display. Alternatively oradditionally, a laser or LED source and scanning system could be used todraw a raster display directly onto the retina of one or more of theuser's eyes. Other possibilities exist as well.

FIG. 1C illustrates another wearable computing system according to anexample embodiment, which takes the form of an HMD 152. The HMD 152 mayinclude frame elements and side-arms such as those described withrespect to FIGS. 1A and 1B. The HMD 152 may additionally include anon-board computing system 154 and an image capture device 156, such asthose described with respect to FIGS. 1A and 1B. The image capturedevice 156 is shown mounted on a frame of the HMD 152. However, theimage capture device 156 may be mounted at other positions as well.

As shown in FIG. 1C, the HMD 152 may include a single display 158 whichmay be coupled to the device. The display 158 may be formed on one ofthe lens elements of the HMD 152, such as a lens element described withrespect to FIGS. 1A and 1B, and may be configured to overlaycomputer-generated graphics in the user's view of the physical world.The display 158 is shown to be provided in a center of a lens of the HMD152, however, the display 158 may be provided in other positions, suchas for example towards either the upper or lower portions of thewearer's field of view. The display 158 is controllable via thecomputing system 154 that is coupled to the display 158 via an opticalwaveguide 160.

FIG. 1D illustrates another wearable computing system according to anexample embodiment, which takes the form of a monocular HMD 172. The HMD172 may include side-arms 173, a center frame support 174, and a bridgeportion with nosepiece 175. In the example shown in FIG. 1D, the centerframe support 174 connects the side-arms 173. The HMD 172 does notinclude lens-frames containing lens elements. The HMD 172 mayadditionally include a component housing 176, which may include anon-board computing system (not shown), an image capture device 178, anda button 179 for operating the image capture device 178 (or usable forother purposes). Component housing 176 may also include other electricalcomponents or may be electrically connected to electrical components atother locations within or on the HMD. HMD 172 also includes a BCT 186.

The HMD 172 may include a single display 180, which may be coupled toone of the side-arms 173 via the component housing 176. In an exampleembodiment, the display 180 may be a see-through display, which is madeof glass or another transparent or translucent material, such that thewearer can see their environment through the display 180. Further, thecomponent housing 176 may include the light sources (not shown) for thedisplay 180 or optical elements (not shown) to direct light from thelight sources to the display 180. As such, display 180 may includeoptical features that direct light that is generated by such lightsources towards the wearer's eye, when HMD 172 is being worn.

In a further aspect, HMD 172 may include a sliding feature 184, whichmay be used to adjust the length of the side-arms 173. Thus, slidingfeature 184 may be used to adjust the fit of HMD 172. Further, an HMDmay include other features that allow a wearer to adjust the fit of theHMD, without departing from the scope of the invention.

FIGS. 1E to 1G are simplified illustrations of the HMD 172 shown in FIG.1D, being worn by a wearer 190. As shown in FIG. 1F, when HMD 172 isworn, BCT 186 is arranged such that when HMD 172 is worn, BCT 186 islocated behind the wearer's ear. As such, BCT 186 is not visible fromthe perspective shown in FIG. 1E.

In the illustrated example, the display 180 may be arranged such thatwhen HMD 172 is worn, display 180 is positioned in front of or proximateto a user's eye when the HMD 172 is worn by a user. For example, display180 may be positioned below the center frame support and above thecenter of the wearer's eye, as shown in FIG. 1E. Further, in theillustrated configuration, display 180 may be offset from the center ofthe wearer's eye (e.g., so that the center of display 180 is positionedto the right and above of the center of the wearer's eye, from thewearer's perspective).

Configured as shown in FIGS. 1E to 1G, display 180 may be located in theperiphery of the field of view of the wearer 190, when HMD 172 is worn.Thus, as shown by FIG. 1F, when the wearer 190 looks forward, the wearer190 may see the display 180 with their peripheral vision. As a result,display 180 may be outside the central portion of the wearer's field ofview when their eye is facing forward, as it commonly is for manyday-to-day activities. Such positioning can facilitate unobstructedeye-to-eye conversations with others, as well as generally providingunobstructed viewing and perception of the world within the centralportion of the wearer's field of view. Further, when the display 180 islocated as shown, the wearer 190 may view the display 180 by, e.g.,looking up with their eyes only (possibly without moving their head).This is illustrated as shown in FIG. 1G, where the wearer has movedtheir eyes to look up and align their line of sight with display 180. Awearer might also use the display by tilting their head down andaligning their eye with the display 180.

FIG. 2 is a simplified block diagram a computing device 210 according toan example embodiment. In an example embodiment, device 210 communicatesusing a communication link 220 (e.g., a wired or wireless connection) toa remote device 230. The device 210 may be any type of device that canreceive data and display information corresponding to or associated withthe data. For example, the device 210 may take the form of or include ahead-mountable display, such as the head-mounted devices 102, 152, or172 that are described with reference to FIGS. 1A to 1G.

The device 210 may include a processor 214 and a display 216. Thedisplay 216 may be, for example, an optical see-through display, anoptical see-around display, or a video see-through display. Theprocessor 214 may receive data from the remote device 230, and configurethe data for display on the display 216. The processor 214 may be anytype of processor, such as a micro-processor or a digital signalprocessor, for example.

The device 210 may further include on-board data storage, such as memory218 coupled to the processor 214. The memory 218 may store software thatcan be accessed and executed by the processor 214, for example.

The remote device 230 may be any type of computing device or transmitterincluding a laptop computer, a mobile telephone, head-mountable display,tablet computing device, etc., that is configured to transmit data tothe device 210. The remote device 230 and the device 210 may containhardware to enable the communication link 220, such as processors,transmitters, receivers, antennas, etc.

Further, remote device 230 may take the form of or be implemented in acomputing system that is in communication with and configured to performfunctions on behalf of client device, such as computing device 210. Sucha remote device 230 may receive data from another computing device 210(e.g., an HMD 102, 152, or 172 or a mobile phone), perform certainprocessing functions on behalf of the device 210, and then send theresulting data back to device 210. This functionality may be referred toas “cloud” computing.

In FIG. 2, the communication link 220 is illustrated as a wirelessconnection; however, wired connections may also be used. For example,the communication link 220 may be a wired serial bus such as a universalserial bus or a parallel bus. A wired connection may be a proprietaryconnection as well. The communication link 220 may also be a wirelessconnection using, e.g., Bluetooth® radio technology, communicationprotocols described in IEEE 802.11 (including any IEEE 802.11revisions), Cellular technology (such as GSM, CDMA, UMTS, EV-DO, WiMAX,or LTE), or Zigbee® technology, among other possibilities. The remotedevice 230 may be accessible via the Internet and may include acomputing cluster associated with a particular web service (e.g.,social-networking, photo sharing, address book, etc.).

III. EXAMPLE SYSTEM

Example embodiments of a device capable of dynamically controlling atleast one setting of an image device based, at least in part, onbiometric data will now be described in greater detail. In general, anexample device may be implemented as part of a wearable orbody-mountable image-capture device (also referred to as a wearablecomputing device), such as a head-mountable device (HMD), as describedabove, a mobile phone with camera or stand-alone digital camera that maybe attached or mounted to a user, such as by an arm-band, wrist band,wrist mount, or a chest-mount system, among other possibilities.Further, an example device may include a non-transitory computerreadable medium, which has program instructions stored thereon that areexecutable by a processor to provide the functionality described herein.An example imaging device may also be implemented as part of or take theform of a wearable computer or mobile phone, or a subsystem of such adevice, which includes such a non-transitory computer readable mediumhaving such program instructions stored thereon.

FIG. 3 is a simplified block diagram of a system 300 including capturedevice 310, processing device 330 and at least one sensor 338, accordingto an example embodiment. The capture device 310 may be any type ofdevice that can capture and process image or video data. For example,the capture device 310 may take the form of or be implemented as part ofa head-mountable display, such as the head-mounted devices 102, 152, or172 that are described with reference to FIGS. 1A to 1G.

The capture device 310 may include an image capture device 312 thatincludes a video camera 316 configured to capture video data, and acontrol system 314. Control system 314 may include a control unit 318, aprocessor 320 and a memory 322. The control unit 318 may be configured,at least in part, to cause the video camera 316 to capture video dataand to control at least one image capture setting of the video camera316. Video data captured by the video camera 316 in the image capturedevice 312 may be used to generate a video file. The processor 320 maybe any type of processor, such as a micro-processor or a digital signalprocessor, for example. The processor 320 may receive biometric datafrom one or more sensors 338. At least one clock 326, or other timingdevice, may also be provided for synchronizing the video camera 316 andsensor 338 outputs. Memory 322 may be provided as a non-transitorycomputer-readable media such as computer-readable media that stores datafor short periods of time like register memory, processor cache, orrandom access memory (RAM). Memory 322 may also be provided as anon-transitory computer-readable media that stores program code or datafor longer periods of time, such as secondary or persistent long termstorage, like read only memory (ROM), optical or magnetic disks, orcompact-disc read only memory (CD-ROM), for example. Any other volatileor non-volatile data storage system may also be used. Memory 322 mayalso be considered a computer-readable storage medium, for example, or atangible storage device.

Sensor 338 may include any type of sensor capable of detecting biometricdata from a user of the capture device 310. Biometric data may includeany data comprising or related to the sensory, physiological,biological, behavioral, health, motion, context or other measurablecharacteristics, phenomena or response of a living subject or a functionof the living subject. Biometric data may be gathered from directmeasurement of an aspect or characteristic of the body, or any dataindicative of the movement or context of a body. For example, biometricdata may include heart rate, pulse rate, respiration rate, bodytemperature, perspiration, eye movements, blinking, muscle flinches ortension, strength of hand grip, etc. Biometric contextual data mayinclude any data indicative of the environment, location, ambientconditions, and status of the user, including location such as GPS (orother location-tracking) data, elevation, type of movement (i.e.walking, driving, flying, etc.), type of activity (running, biking,swimming, etc.), weather, ambient temperature, ambient light intensity,time of day, height of a user, weight of a user, age of a user, etc.Biometric motion-related data may include speed or direction of travel,change in altitude, acceleration, cadence, direction or intensity ofmovement of the body, orientation of the body, gravitational andinertial forces acting on the body, rotation, etc.

In one aspect, the one or more sensors 338 for sensing biometric datamay include one or more of a heart rate monitor, a respiration ratesensor, a thermometer, a perspiration sensor (such as a galvanic skinresponse sensor), a microphone, a decibel meter, a bone conductingtransducer, an electromyograph, a strain gauge, an eye-detection sensor,a pulse oximeter, a Doppler device, an eye-tracking device, etc. Sensors338 for sensing motion-related biometric data may include one or moremovement or positioning sensors, such as, an accelerometer, an inertialmeasurement unit (IMU), a proximity sensor; a microphone; a gyroscope, amagnetometer, an optical sensor, an ultrasonic sensor, an odometer, anda pedometer. Such motion sensors may also detect the position andorientation of a user of the device without necessarily detecting“movement.” Further, the one or more sensors 338 for detectingcontextual-related biometric data may include one or more of alocation-determination sensor, a light intensity sensor, a clock and asensor configured to receive an input from a user (such as whether sheis flying, travelling in a car, walking, etc.).

Sensor 338 may be integral to or separate from the capture device 310,as shown in FIG. 3. For example, the one or more sensors may beintegrated on an HMD, or may be remote to the HMD (such as biometricsensors placed on other portions of the body or in communication withthe body). The one or more sensors 338 may also be provided on acomputing device remote from the HMD (such as a remote device such as asmartphone having location tracking and internet capabilities).

The processing device 330 may include an image processing system 332 anda memory 334 having program instructions 336 stored thereon. The imageprocessing system 332 may be any type of processor, such as amicro-processor or a digital signal processor, for example. Memory 334may be provided as a non-transitory computer-readable media such ascomputer-readable media that stores data for short periods of time likeregister memory, processor cache, or random access memory (RAM). Memorymay also be provided as a non-transitory computer-readable media thatstores program code or data for longer periods of time, such assecondary or persistent long term storage, like read only memory (ROM),optical or magnetic disks, or compact-disc read only memory (CD-ROM),for example. Any other volatile or non-volatile data storage system mayalso be used. Memory 334 may also be considered a computer-readablestorage medium, for example, or a tangible storage device.

An interface 340 is configured to receive video data generated by theimage capture device 340, such as by video camera 316. Biometric datadetected by the one or more sensors 338 may be received by interface 342of the processing device 330. The processing device 330 is configured toprocess the received video data, with at least one type of imageprocessing, based, at least in part, on the received biometric data, togenerate edited video data. The video data and the biometric data aregenerated synchronously and may be appropriately labelled with timesynchronization data by the clock 326, for example, indicating a timingrelationship between the biometric data and the video data. Processingof the video data by the processing device 330 may further be based onthe time synchronization data.

The processing device 330 may be implemented as part of or on the sameplatform as the capture device 310, or may be separate from the capturedevice 310, as shown in FIG. 3. For example the processing device may beimplemented as part of or take the form of a head-mountable device (HMD)along with the capture device 310. Alternatively, the processing device330 may also be provided on a computing device remote from the HMD (suchas a remote computing device such as a personal computer, laptopcomputer, tablet computer, or cellular phone). The processing device 330may also be implemented as or take the form of a cloud server. In someembodiments, processing device 330 may receive the video, sensor orsynchronization data wirelessly according to one or more wirelessstandards or protocols, such as, but not limited to, RFID, Bluetooth,Wi-Fi, ZigBee, WiMax, or a Wireless Wide Area Network (e.g., TDMA, CDMA,GSM, UMTS, EV-DO, LTE), etc. In other embodiments, processing device 330may receive the video, sensor and synchronization data by one or morewired protocols such as, but not limited to, a Universal Serial Busprotocol, a Registered Jack protocol (e.g., RJ-25), or a wired LocalArea Network protocol (e.g., Ethernet). In other examples, video, sensorand synchronization data may be received by the processing device from aportable storage device such as a memory card, flash drive, or zipdrive.

In example embodiments, biometric data collected by the one or moresensors 338 may be used by the system 300 in at least two ways to createa more interesting or sensory-representative video composition. In oneaspect, the biometric data may be used to control at least oneimage-capture setting of the image capture device 312, which may includecontrolling image capture software or hardware on the image capturedevice 312. The image capture properties may include, at least, the rateof frame capture and the resolution (e.g., image or spatial). In anotheraspect, biometric data received from the one or more sensors 338 may beused for assisted post-capture image or audio editing by the processingdevice 330. Biometric data from the one or more sensors 338 and otherinputs may be synchronously recorded with the video and audio data andmay essentially “tag” the data with processing cues. These “tags” may beused post-capture to automatically edit or suggest edits to the imageand audio data. The image processing applied by the image processingsystem 332 may include, but is not limited to, fast motion effects, slowmotion effects, blur effects, color calibration, light intensitycalibration, fade-in or fade-out effects, frame cutting, frame rateconversion, white point conversion, color space conversion, noisereduction, detail enhancement, edge enhancement, and motioncompensation.

The manner in which the image capture properties of the image capturedevice 312 are controlled and the manner in which the image processingis applied by the image processing system 332, based on the biometricdata, may depend on qualitative and quantitative properties of thereceived biometric data. The control system 314 of the capture device310 and the image processing system 332 of the processing device 330 maybe configured to control image capture properties and apply imageprocessing, respectively, based on certain logic programmed on thedevices. The logic may dictate that, upon receipt of a certain type ofbiometric data and at a certain level, the image processing systemshould apply a certain type of image processing and in a certain manner.This logic may be based on certain knowledge, observations, conventions,standards or preferences that may allow the ultimate video data tobetter reflect the emotions, perceptions and senses of the user orcreate a more interesting composition. For example, certain observationsmay be made that biometric data reflecting elevated heart, respirationor pulse rate, increased perspiration, and increased cadence of movementindicates that the wearer or user of the device is engaged in a highlevel of activity or is experiencing excitement. Certain conventions maysuggest that these experiences and perceptions may be translated intothe captured video and audio data by, for example, replaying the videoor audio data in slow motion.

To achieve one or more of these functions, the control system 314 of thecapture device 310 may be configured to receive biometric data from theone or more sensors 338 and, while the image-capture device 312 iscapturing video data, control at least one image-capture setting of theimage capture device 312 based, at least in part, on the biometric data.The at least one image-capture setting, which may include the rate offrame capture and the resolution of, for example, the video camera 316,affects the captured video data. The control system 314 may further beconfigured to cause the sensors 338 to generate the biometric data whilethe image-capture device 312 captures the video data.

Specifically, in operation, the processor 320 may execute one or moreprogram instructions 324 stored in memory 322, which may include certainprogram logic. Execution of these program instructions 324 by theprocessor 320 may cause the image capture device 312 to capture videodata, such as with video camera 316. The instructions may further causethe capture device 310 to receive biometric data from the one or moresensors 338. The biometric data may be generated synchronously with thevideo data generated by the video camera 316. Clock 326 may providetime-synchronization data that indicates a timing relationship betweenthe biometric data and the video data. The processor 320 may receive thebiometric data from the one or more sensors 338 and execute the programinstructions 324, which may relate to when and how the receivedbiometric data is to be used by the control system 314. While theimage-capture device 316 continues to capture video data, execution ofthe program instructions 324 by the processor may cause the control unit318 of the control system 314 to control at least one image-capturesetting of the image capture device 312.

In an example embodiment, the control unit 318 is configured to controlat least one image capture setting based, at least in part, on thebiometric data from the sensor 338. Data taken from the sensor 338, suchas heart rate, may be used to control software or hardware of thecapture device 310. For example, the control unit 318 may be configuredto increase the rate of image frame capture of the video camera 316 uponreceiving biometric data indicative of a higher than normal heart rate.An increased heart rate may indicate that the user is operating at ahigh level of activity, is frightened or excited, or is in some othersituation in which a high frame rate or higher resolution may bedesirable. In response, the control unit 318 may increase the resolutionor frame rate of the video camera 316. In another example, uponreceiving biometric data indicative of falling, such as from an IMUlocated on or in communication with capture device 310, the control unit318 may increase the rate of image frame capture of the video camera316. This portion of increased frame rate video may later be used tocreate a “slow motion” section of the resulting video file, therebyhighlighting the user's free-fall. In a further example, the controlsystem may be configured to increase the rate of image frame capturebased, at least in part, on receiving biometric data indicative of ahigher than normal respiration rate, which may also indicate increasedactivity or excitement. Certain thresholds relating to the biometricdata, such as normal heart and respiration rates, may be stored inmemory 322. Many other inferences may be made between the collectedbiometric data and the activity level, emotions or sensory perceptionsof users of the capture device 310.

To further achieve one or more of the functions described above, theimage processing system 332 may be used, post-capture, to enhance thevideo data by applying one or more image processing techniques. Inoperation, the image processing system 332 may execute one or moreprogram instructions 336 stored in memory 334 on the processing device330. Execution of these program instructions 336 by the image processingsystem 332 may cause the processing device 330 to receive video datagenerated by an image capture device 312, via interface 340. Further,the image processing system may cause the processing device 330 toreceive, from the one or more sensors 338, biometric data of a user ofthe image capture device 312, via interface 342.

The image processing system 332 may further be configured to apply imageprocessing to the video data based, at least in part, on the biometricdata, and to generate edited video data therefrom. Visual emphasis (orde-emphasis) may be achieved by adjusting the speed of the video, colorcalibration, light intensity calibration, white point conversion, colorspace conversion or by applying blur effects, fade-in or fade-outeffects, frame cutting, frame rate conversion, noise reduction, detailenhancement, edge enhancement, and motion compensation.

For example, biometric data collected from a heart-rate monitor (HRM)indicating a normal or slower than normal heart rate may cause the imageprocessing system 332 to increase the speed of the video data togenerate edited video data. A normal or lower than normal heart rate mayindicate that the user may not be engaged in rigorous movement and maynot be experiencing anything particularly exciting. In such cases, wherethe captured video may be considered of less interest, it may bedesirable to increase the speed of the video data, for example, as fastplayback. Alternatively, biometric data indicating a higher than normalheart rate or sharp increase in the wearer's heart rate may cause theimage processing system 332 to slow-down that segment of recorded videoto create visual emphasis. In another example, the image processingsystem may be configured to decrease the speed of the video data togenerate edited video data based, at least in part, on receivingbiometric data indicative of falling or on receiving biometric dataindicative of a higher than normal respiration rate. Visual emphasis mayalso be achieved by changing the color saturation of the video data. Forexample, the image processing system 332 may be configured to adjust thecolor saturation towards yellow based on receiving biometric dataindicative of speed of travel.

Further, the image processing applied to the video data may include oneor more audio effects, such as audio overlay, noise removal, noiseamplification, noise suppression, audio volume adjustment, pitch shift,or time stretching effects. For example, the image processing system maybe configured to apply an audio overlay such as music or a heartbeatsound to the video data. The video data may include audio data. In otherexamples, the image processing system may be configured to suppress oramplify background audio noise present in the video data, such as windnoise or motorcycle noise.

The image processing system 332 may be configured to automatically applythe one or more image processing techniques to the captured video data.In other aspects, the image processing system 332 may be configured tosuggest that an image processing be applied based on the biometric dataor otherwise “tag” portions of the video data based on the collectedbiometric data for subsequent manual editing.

Biometric data may be continuously received by the processor 320 and theimage processing system 332 and analyzed to determine whether one ormore settings of the image capture device should be controlled or if oneor more image processing techniques should be applied by the processingdevice. Similarly, the biometric data can be used to determine if animage capture setting should be returned to normal or if one or moreimage processing techniques should be ceased.

Further, one of skill in the art will recognize that biometric data ofdifferent types received from different types of sensors 338 may be usedin conjunction by the capture device 310 and the processing device 330to achieve their respective functions. For example, biometric datareceived from a heart-rate monitor indicating an increased heart ratemay be checked against biometric data received from one or more motionsensors, such as an optical sensor on a user's bike, to determine if theuser is still moving (i.e., still pedaling the bike) to decide if animage processing should be applied. In addition, motion-relatedbiometric data may be used in conjunction with context-related biometricdata gathered by one or more sensors. For example, if the systemreceives biometric data, such as by direct input from the user,indicating that the user is travelling by car, the system may determinethat a high speed of travel by the user does not necessarily indicatethat an image capture setting should be controlled or image processingshould be applied to the video data. Similarly, biometric dataindicating increased perspiration may be compared to context-relatedbiometric data indicating a high ambient temperature.

In operation, a user of the image capture device 310 enters videorecording mode by, for example, manual input or by speaking a verbalcommand. As part of the video recording mode, a user interface or one ormore graphics may be provided on a display. Where the image capturedevice 310 takes the form of or includes an HMD, the display may be, forexample, an optical see-through display, an optical see-around display,or a video see-through display. The user interface or graphics mayindicate that one or more types of biometric data are being received bythe image capture device 310. For example, the user interface or graphicmay include a heart icon to indicate that heart rate data is beingreceived from a heart-rate sensor. In addition, the user interface orgraphics may provide a real-time indication of the biometric data beingreceived. For example, the user interface or graphic may also include anumber indicating the wearer's heart rate as sensed by the correspondingsensor. The displayed graphics or interface may be time-synced with thereceived biometric sensor data such that the numerical value is updatedin real time as new sensor data is received.

Additionally or alternatively, an overlaid user interface or graphic 410may be added to the captured video data by the image processing system332 as part of the image processing to generate edited video data. Theuser interface or graphic may include images or icons indicating thetype of biometric data that was sensed concurrently with the capturedvideo. For example, a heart icon may indicate that heart rate data wassensed, a mountain icon may indicate that altitude data was sensed, or awater droplet icon may indicate that perspiration data was sensed. Inaddition, the user interface or graphic may include an indication orquantification, such as a numerical value, of the sensed biometric data,such as an indication of the wearer's heart rate, respiration rate,speed or cadence of movement, elevation, etc. The edited video data maybe output from the image processing system 332 to a display 344 fordisplaying the edited video data.

As shown in FIG. 4, which illustrates a screenshot of a video display344, when played back on the display 344, the edited video data willinclude the user interface or graphic 410. Further, as the sensedbiometric data is time synced with the captured video data, thebiometric data will also be time synced with the edited video data suchthat the indication of biometric data overlaid as part of the userinterface or graphic will update as the edited video data is replayed.The user interface or graphic may be applied as part of the imageprocessing to the edited video data independently of the user interfaceor graphic being displayed on the HMD display.

As part of the image processing, music or sounds may be applied to thevideo data to generate edited video data. For example, a heartbeat soundmay be applied to the video data. The user interface or graphic andsound may be applied to the video data prior to other types of the imageprocessing, such as fast or slow motion effects. Accordingly, theapplied sound also be sped up or slowed down as part of the imageprocessing along with the captured video data. While illustrated as aseparate element, the display 344 may be provided on the same platformas image processing system 332.

IV. EXAMPLE METHODS

FIG. 5 is a flow chart illustrating a method 500, according to anexample embodiment. Illustrative methods, such as method 500, may becarried out in whole or in part by an HMD, such as the head-mountabledevices shown in FIGS. 1A to 1G. Method 500 may be carried out bycomponents of an HMD, such as a capture device 310. Such a capturedevice may include, for example, an image capture device and a controlsystem having a control unit, a processor and program instructionsstored on a non-transitory computer-readable medium. However, an HMD'scapture device may additionally or alternatively include othercomponents. Further, an example method or portions thereof may becarried out by additional components of an HMD. Yet further, an examplemethod, or portions thereof, may be carried out by a capture device thatis in communication with an HMD. An example method may also be carriedout by other types of computing devices or combinations of computingdevices including other types of wearable computing devices as well.

In a first step, video data is received by a computing device from animage-capture device, such as image capture device 312. (510). Inaddition, biometric data, which is generated synchronously with thevideo data, is received by the computing device from one or moresensors. (520). As described above, the one or more sensors (such assensors 338) may be any sensor configured to detect biometric data,which may include any data related to the sensory, physiological,behavioral, health, movement, context or other measurablecharacteristics of a user of the image capture device or a function ofthe user. While the image-capture device is capturing video data, atleast one image-capture setting of the image capture device may becontrolled based, at least in part, on the biometric data, therebyaffecting the captured video data. (530). The at least one image capturesetting includes rate of image frame capture, and resolution.

FIG. 6 is a flow chart illustrating a method 600, according to anexample embodiment. Illustrative methods, such as method 600, may becarried out in whole or in part by a computing device. The computingdevice may be an HMD, such as the head-mountable devices shown in FIGS.1A to 1G. Method 600 may be carried out by components of an HMD, such asa processing device 330. Such a processing device may include, forexample, an image processing system and program instructions stored on anon-transitory computer-readable medium. However, an HMD's processingdevice may additionally or alternatively include other components.Further, an example method or portions thereof may be carried out byadditional components of an HMD. Yet further, an example method, orportions thereof, may be carried out by a processing device that isremote from an HMD, such as in system 300. An example method may also becarried out by other types of computing devices or combinations ofcomputing devices including other types of wearable computing devices aswell.

In a first step, video data generated by an image capture device isreceived by a computing device. (610). Biometric data of a user of theimage capture device is received by the computing device from one ormore sensors. (620). The biometric data is generated synchronously withthe video data. Image processing is applied to the video data based, atleast in part, on the biometric data to generate edited video data(630). As described above, the image processing may include, but is notlimited to: fast motion effects, slow motion effects, blur effects,color calibration, light intensity calibration, fade-in or fade-outeffects, frame cutting, frame rate conversion, white point conversion,color space conversion, noise reduction, detail enhancement, edgeenhancement, and motion compensation. The edited video data may beoutput to a user interface, such as a video screen, as shown in FIG. 4.

Time-synchronization that indicates a timing relationship between thebiometric data and the video data may also be received. Thissynchronization data may allow the image processing system to align theuser's biometric signals with the video that was captured at that timeand apply biometrically-relevant image processing to the video data.Accordingly, the image processing system may further base the imageprocessing of the video data on the time-synchronization data.

Wearable devices implementing the methods and or devices described abovehave many possible applications where adjusting image capture settingsor applying relevant image processing to the video data may bedesirable. One such application may be in the field of extreme sportswhere users may desire to take point-of-view action video of their ownperspective. Take, for example, a person using an HMD having a capturedevice, such as capture device 310, to capture point-of-view video whilemountain biking. Sensors in communication with the capture device, willdetect biometric signals from the user, such as heart rate, pedalingspeed, level of perspiration, and level of auditory noise. Upon receiptof biometric data indicating that the user has an increased heart rateand increased auditory response, the capture device may increase therate of frame capture of the video camera, from, for example, 30 framesper second to 120 frames per second. This action by the capture devicemay be based on logic that an increased heart rate and an increasedauditory response (e.g., yelling or shouting) indicate high emotion orexcitement by the user, which may further indicate exciting, valuable orinteresting video data. By increasing the frame rate, this portion ofvideo data may subsequently be used to create a slow-motion segment ofthe video file, which may add visual interest and emotional context tothe video. The video file generated from this recorded session willinclude some portion at 30 frames per second and some portions at 120frames per second.

Further, image processing may further be applied to the video data by aprocessing device, such as device 330. In the example described above,an image processing system may, in response to the biometric data of theuser indicating increased heart rate and increased auditory response,apply slow motion image processing or color enhancement imageprocessing, as examples. The processing device may also suggest to auser that certain image processing be applied to the video data.Alternatively, based on biometric data received from one or more opticalmotion sensors positioned on the user's bike indicating that the userhas ceased pedaling for an extended period of time, the processingdevice may discard (or cut) certain frames. This action by theprocessing device may be based on logic that a cessation in movementindicates relatively unexciting or uninteresting video data. Many otherapplications are contemplated.

V. CONCLUSION

In the figures, similar symbols typically identify similar components,unless context indicates otherwise. The illustrative embodimentsdescribed in the detailed description, figures, and claims are not meantto be limiting. Other embodiments can be utilized, and other changes canbe made, without departing from the scope of the subject matterpresented herein. It will be readily understood that the aspects of thepresent disclosure, as generally described herein, and illustrated inthe figures, can be arranged, substituted, combined, separated, anddesigned in a wide variety of different configurations, all of which areexplicitly contemplated herein.

With respect to any or all of the message flow diagrams, scenarios, andflow charts in the figures and as discussed herein, each step, block orcommunication may represent a processing of information or atransmission of information in accordance with example embodiments.Alternative embodiments are included within the scope of these exampleembodiments. In these alternative embodiments, for example, functionsdescribed as steps, blocks, transmissions, communications, requests,responses, or messages may be executed out of order from that shown ordiscussed, including in substantially concurrent or in reverse order,depending on the functionality involved. Further, more or fewer steps,blocks or functions may be used with any of the message flow diagrams,scenarios, and flow charts discussed herein, and these message flowdiagrams, scenarios, and flow charts may be combined with one another,in part or in whole.

A step or block that represents a processing of information maycorrespond to circuitry that can be configured to perform the specificlogical functions of a herein-described method or technique.Alternatively or additionally, a step or block that represents aprocessing of information may correspond to a module, a segment, or aportion of program code (including related data). The program code mayinclude one or more instructions executable by a processor forimplementing specific logical functions or actions in the method ortechnique. The program code or related data may be stored on any type ofcomputer-readable medium, such as a storage device, including a diskdrive, a hard drive, or other storage media.

The computer-readable medium may also include non-transitorycomputer-readable media such as computer-readable media that stores datafor short periods of time like register memory, processor cache, orrandom access memory (RAM). The computer-readable media may also includenon-transitory computer-readable media that stores program code or datafor longer periods of time, such as secondary or persistent long termstorage, like read only memory (ROM), optical or magnetic disks, orcompact-disc read only memory (CD-ROM), for example. Thecomputer-readable media may also be any other volatile or non-volatilestorage systems. A computer-readable medium may be considered acomputer-readable storage medium, for example, or a tangible storagedevice.

Moreover, a step or block that represents one or more informationtransmissions may correspond to information transmissions betweensoftware or hardware modules in the same physical device. However, otherinformation transmissions may be between software modules or hardwaremodules in different physical devices.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether programs orfeatures collect user information (e.g., information about a user'ssocial network, social actions or activities, profession, a user'spreferences, or a user's current location), or to control whether and/orhow to receive content from the content server that may be more relevantto the user. In addition, certain data may be treated in one or moreways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by a content server.

We claim:
 1. A computing device, comprising: an interface configured toreceive video data that is generated by an image capture device; aninterface configured to receive, from one or more sensors, biometricdata of a user of the device generated during capture of the receivedvideo data, wherein the biometric data corresponds in time with thevideo data; and a processing system operable to: determine that ameasurable characteristic of a first portion of the biometric datadiffers by a threshold amount from a corresponding measurablecharacteristic of a second portion of the biometric data; automaticallyapply image processing to the video data to generate edited video data,wherein the image processing comprises, in response to a determinationthat the measurable characteristic of the first portion differs by thethreshold amount from the second portion, applying a first colorparameter adjustment to the first portion of video data, but not to thesecond portion of the video data; and generate, and store to datastorage for subsequent playback, a video file comprising the editedvideo data.
 2. The computing device of claim 1, wherein applying thefirst color parameter adjustment to the first portion of video data, butnot to the second portion of the video data, comprises: adjusting one ormore of the following color parameters for the first portion of thevideo data: (a) a saturation parameter, (b) a hue parameter colorcalibration, (c) an intensity parameter, (d) a white balance parameter,or (e) a color-space conversion parameter.
 3. The computing device ofclaim 1, wherein the biometric data comprise heartrate data indicating ahigher heartrate at a time corresponding to the first portion of thevideo data than at a time corresponding to the second portion of thevideo data, and wherein the application of the first color parameteradjustment to the first portion of video data, but not to the secondportion of the video data, comprises: adjusting a first color parameterfor the first portion of video data so as to visually emphasize thefirst portion of the video data as compared to the second portion of thevideo data.
 4. The computing device of claim 3, wherein the adjustmentto the first color parameter for the first portion of video datacomprises an adjustment to hue, saturation, or both towards a particularcolor.
 5. The computing device of claim 1, further comprising: aninterface configured to receive time-synchronization data that indicatesa timing relationship between the biometric data and the video data,wherein the image processing of the video data is further based on thetime-synchronization data.
 6. The computing device of claim 1, whereinthe computing device is implemented as part of or takes the form of ahead-mountable device (HMD).
 7. The computing device of claim 1, whereinthe computing device is implemented as or takes the form of a cloudserver.
 8. The computing device of claim 1, wherein the biometric datacomprises one or more of the following types of data: (a) heart ratedata, (b) respiration rate data, (c) body temperature data, (d) level ofperspiration data, (e) muscle movement data, (f) eye movement data, or(h) speech data.
 9. The computing device of claim 5, wherein the one ormore sensors comprise one or more of the following types of sensors: (a)a heart rate monitor, (b) a respiration rate sensor, (c) a thermometer,(e) a perspiration sensor, (f) a microphone, (g) a decibel meter, (h) abone conducting transducer, (i) an electromyograph, (j) a strain gauge,or (k) an eye-detection sensor.
 10. A computing device, comprising: aninterface configured to receive video data that is generated by an imagecapture device; an interface configured to receive, from one or moresensors, biometric data of a user of the device generated during captureof the received video data, wherein the biometric data corresponds intime with the video data; and a processing system operable to: determinethat a measurable characteristic of a first portion of the biometricdata differs by a threshold amount from a corresponding measurablecharacteristic of a second portion of the biometric data; automaticallyapply image processing to the video data to generate edited video data,wherein the image processing comprises, in response to a determinationthat the measurable characteristic of the first portion differs by thethreshold amount from the second portion, applying at least one visualeffect to the first portion of video data, but not to the second portionof the video data; and generate, and store to data storage forsubsequent playback, a video file comprising the edited video data. 11.The computing device of claim 10, wherein the at least one visual effectcomprises one or more of the following visual effects: (a) a fade-ineffect, (b) a fade-out effect, (c) noise reduction, (d) detailenhancement or sharpening, (e) edge enhancement, or (f) motioncompensation.
 12. The computing device of claim 10, further comprisingone or more of the following types of sensors: (a) an accelerometer, (b)an inertial measurement unit, (c) a proximity sensor; (d) a microphone;(e) a gyroscope, (f) a magnetometer, (g) an optical sensor, (h) anultrasonic sensor, (i) an odometer, and (j) a pedometer.
 13. Thecomputing device of claim 10, wherein the image processing system isfurther configured to: apply image processing based, at least in part,on additional data comprising one or more of the following types ofdata: (a) location, (b) ambient light intensity, (c) time of day, (d) auser's mode of travel, (e) a type of activity a user is participatingin, (f) height of a user, (g) weight of a user, and (h) age of a user.14. The computing device of claim 10, wherein the one or more sensorscomprise one or more contextual sensors from the following contextualsensors: (a) a location-tracking sensor, (b) light intensity sensor, (c)a clock, or (d) a sensor configured to receive an input from a user. 15.The computing device of claim 10, wherein the image processing comprisesat least one of: (a) fast motion effects, (b) slow motion effects, (c)blur effects, (d) color calibration, (e) light intensity calibration,(f) fade-in or fade-out effects, (g) frame cutting, (h) frame rateconversion, (i) white point conversion, (j) color space conversion, (k)noise reduction, (l) detail enhancement, (m) edge enhancement, or (n)motion compensation.
 16. The computing device of claim 10, wherein theimage processing system is configured to decrease a speed of the videodata to generate edited video data based, at least in part, on receivingbiometric data indicative of a higher than normal respiration rate. 17.The computing device of claim 10, wherein the image processing system isconfigured to adjust a color saturation of the video data to generateedited video data based, at least in part, on receiving biometric dataindicative of speed of travel.
 18. The computing device of claim 10,wherein the image processing comprises one or more audio effectsincluding: (a) audio overlay, (b) noise removal, (c) noiseamplification, (d) noise suppression, (e) audio volume adjustment, (f)pitch shift, and (g) time stretching.